Skip to content

A run writes its genotype truth beside its records - #49

Merged
MuteJester merged 6 commits into
masterfrom
phase7/genotype-truth-sidecar
Nov 5, 2026
Merged

MuteJester merged 6 commits into
masterfrom
phase7/genotype-truth-sidecar

Conversation

@MuteJester

Copy link
Copy Markdown
Owner

A simulated dataset used to leave this library through two doors: the records
from the result, the genotype from the Genotype object the caller still had
to be holding. Hand the dataset to someone else and it arrived without the
thing it was meant to be scored against. This adds the sidecar that closes
that, and corrects a test-suite catalogue that described the area wrongly.

What is new

result.to_genotype_tsv(path) and CohortResult.to_genotype_tsv(path) write
the genotype as one row per carried allele — subject_id, locus, segment,
gene, allele, chromosome, novel, gene_state — so one sidecar joins
to one records file. Genotype.to_table() / to_tsv() are untouched; this is
a projection of them, not a replacement.

Two decisions worth stating:

  • A deleted gene keeps its rows, with the literal allele (none). "Gene
    absent from the table" and "gene present and deleted on both chromosomes"
    are different truths, and a genotype caller is scored on exactly that
    difference. An empty TSV cell cannot be told apart from a missing value.
  • No per-read novelty column. Whether a read drew a novel allele is a join
    of truth_v_call against the sidecar on (subject_id, allele). The record
    already carries both halves, so a derived column would be a third copy. The
    guide shows the join concretely, run end to end before being written down:
    27 of 50 records when the planted novel allele is the subject's only V
    option, 0 when it is one of 52.

What it refuses, and why that is most of the work

A zero-row truth file beside a million records is a failure that looks like
success, so the writer refuses rather than writing one. Four cases, each
measured:

  • No genotype at all.
  • A permissive genotype that still leaves reference genes unspecified. The
    engine samples those genes over the whole reference, unphased, so their
    alleles reach the records but no row of the file. permissive() alone wrote
    a 66-byte header-only file while the records named five real alleles; with
    one gene pinned it wrote two rows while the records drew 119 alleles
    absent from the file
    . Keyed on what is specified, not on is_permissive,
    because permissive().complete_from_reference() stays permissive while its
    table becomes complete.
  • A cohort where only some subjects carry a genotype — the Nones used to
    be dropped, producing a table short by whole subjects that looked complete.
  • Two subjects sharing one subject_id — their rows would merge under one
    join key and neither subject's truth would be recoverable.

Every refusal happens before the file is opened, so a refusal cannot leave a
partial artifact behind.

The catalogue was wrong, not just stale

germline.genotype declared that the genotype "is not exported" and that
there is "no haplotype or chromosome field". The second half had never been
true: haplotype is recorded by the engine and stamped on every record of a
genotype run, and nine existing tests read it while the catalogue said the
field did not exist.

The reason it survived is the field census — no mode in record_modes
attached a genotype, so subject_id and haplotype were invisible to the two
completeness checks that exist to catch exactly this. That module's own
docstring already describes the same failure for the duplex mode. Adding the
genotype mode immediately reported two exported fields nobody was answerable
for.

The gap is narrowed, not removed. Per-chromosome expression weight is
exported nowhere, so a genotype with chromosome_weights(1.0, 0.0) projects
85 rows claiming haplotype-1 alleles are carried while no record can draw one.
Deleting the gap outright would have over-claimed. germline.genotype moves
no-truth-exported → gap-limited; the pin is now 22 verified / 4
gap-limited / 3 no-truth / 2 consistent.

haplotype is covered on a reference basis by the phasing test, which reads
the drawn chromosome off a genotype it constructed itself. That test also
gained a premise guard: without it the claim held in one direction only, since
a run that never left chromosome 0 satisfies every assertion in it while
proving nothing about chromosome 1.

One documented property was false

A record carries one locus for the whole rearrangement, taken from its V
call. A sidecar row carries the locus of its own gene. On HUMAN_TCRD_IMGT,
whose V pool holds both TRAV and TRDV genes, 134 of 200 records
disagree with the sidecar row for the very allele they were generated from.
The join is on the allele and is unaffected, so the column is now documented
as what it is, with a warning against filtering by it. The test class that was
named for this property — and never looked at a record, which is how it
survived — now reads real records on both a single-locus and a mixed-locus
cartridge.

Verification

  • pytest -n auto -m "not performance and not campaign": 5856 passed, 17
    skipped (5812 before the branch)
  • pytest -m performance, serial: 9 passed
  • pytest tests/acceptance/: 94 passed — the sha256 determinism and
    reproducibility pins are green untouched. No engine or record bytes
    change; the two new methods are called by nothing in a run path.
  • cargo test --all-features: 1735 passed across 6 suites (no engine_rs/
    file is touched)
  • ruff check tests/ src/ tools/: clean
  • inventory_gate.py check --since origin/master --declarations: clean

The completeness suite was proved non-vacuous against three injected defects:
dropping chromosome 1 fails completeness, never flagging novel fails the
novel test, and omitting deleted rows fails both the deletion premise and the
deleted-genes test.

Also in here

The sidecar wrote CRLF beside a records file that pins LF, leaving a stray
carriage return on the last column in any shell pipeline joining the two. Four
of the eight columns had no file-level assertion, including novel, whose
spelling the documented join compares against the literal "True". The
deletion question was documented as answerable per read, which it is not,
since a deleted gene is never drawn from. The fully-deleted premise guard was
satisfied by a hemizygous deletion, which would have left its test asserting
nothing. And Genotype.to_tsv opened without an explicit encoding, so a
non-ASCII subject_id raised on Windows — pre-existing, but the guide now
recommends it beside the new method.

`Genotype.to_table()` is one row per gene with the alleles as list-valued
columns. That shape is wrong for a TSV a benchmark joins against: a list
inside a flat cell is a nested format inside a flat one, and every
consumer writes its own splitter. `genotype_truth_rows` projects it into
one row per (gene, chromosome, carried allele).

`to_table()` is unchanged -- it is a public, tested surface with its own
callers, and this is a projection of it, not a replacement.

Two properties are pinned harder than the shape.

A deleted gene still gets rows. It is the one point where the long form
can lose what the wide form keeps: "absent from the table" and "present
and deleted" are different truths, and a genotype caller is scored on
exactly that difference. Deleted on one chromosome gives one allele row
and one `(none)` row; deleted on both gives two. `gene_state` carries the
zygosity verbatim so the wide form stays recoverable.

The allele cell for a deleted chromosome is the literal token `(none)`,
not an empty cell, because an empty TSV cell cannot be told apart from a
missing value.

`locus_of` is a deliberate port of the engine's `locus_prefix` from
engine_rs/src/airr_record/locus.rs: the uppercased three-character
prefix, accepted only for the seven AIRR loci, empty otherwise. The
sidecar is joined against a records file, and a reader will compare
`locus` across the two; a rule that drifted from the engine's would put
a value in the truth file that the records disagree with. A synthetic
reference gets an empty locus in both places.

An unplaced novel allele contributes no row: defined but never assigned
to a chromosome, so not carried, so not truth about this genome.
`to_genotype_tsv` on SimulationResult and CohortResult. A records file
carries `subject_id` and nothing that says what that subject's genome
was; this is the companion artifact that closes the join.

The refusals carry more weight here than the happy path. A result with
no genotype raises instead of writing a header-only file, because a
zero-row truth file beside a million records is the failure mode that
looks like success -- the export appears to have worked and the
benchmark silently scores against nothing. A genotype with no
subject_id raises for the same reason: the sidecar would be unjoinable
while looking complete. Both checks run before the file is opened, so a
refused export leaves nothing behind rather than a partial artifact
someone later mistakes for truth.

Writes with an explicit encoding, matching `to_fasta` rather than the
platform default.
…decar

The measurement that justified this phase was a probe run once by hand:
over a 2-subject cohort, 0 of 183 distinct V/D/J calls were missing from
the genotype table. This turns it into a standing guarantee over a
3-subject, 450-record cohort.

A sidecar that misses an allele is worse than no sidecar: a benchmark
scoring a caller against it counts a correct call as wrong.

Every assertion is paired with a guard on its own premise, because a
completeness test runs green on a degenerate repertoire -- one subject,
one allele per segment, no deletions. The guards assert that the run
produced records for all three subjects, that more than one allele was
drawn per segment, that the three genotypes differ from each other, that
at least one subject carries a deleted gene, and that no subject carries
the pooled allele set (which would make per-subject truth
indistinguishable from pooled truth).

The deletion is forced rather than hoped for. Whether `Genotype.sample`
happens to delete a gene at these seeds is not something the test should
depend on.

The novel-allele test makes its gene the subject's only V gene. Every V
gene in this cartridge carries the same usage weight, so drawing one
specific gene is a ~0.3% event per record: measured, 10 hits in 3000
records. A test that hoped for the draw would need thousands of records
and still flake.

Verified non-vacuous against three injected defects, each caught by the
test written for it: dropping chromosome 1 fails completeness; never
flagging novel fails the novel test; omitting deleted rows fails both
the deletion premise and the deleted-genes test.
`germline.genotype` carried a truth_gap stating that the genotype "is not
exported" and that there is "no haplotype or chromosome field". The second
half had never been true: the engine stamps `haplotype` -- the chromosome
the rearrangement drew from -- on every record of a genotype run, and
`subject_id` beside it. Nine existing tests read `record["haplotype"]`
while the catalog said the field did not exist.

The reason the contradiction survived is the field census. No mode in
`record_modes` attached a genotype, so `subject_id` and `haplotype` were
invisible to `test_no_exported_field_is_owned_by_nobody` and to
`test_every_truth_field_is_a_field_some_mode_actually_produces` -- the same
blind spot the duplex mode was added to fix, and the module's own docstring
says so about that one.

So the census gains a genotype mode first. It immediately reported two
exported fields nobody is answerable for, which is the check doing its job
on a defect it had been unable to see. `germline.genotype` now claims all
three of its fields, the gap is gone (the carried-allele set is exported by
`to_genotype_tsv` as of this branch), and what replaces it is a note saying
where that truth lives and why per-read novelty is a join rather than a
column.

`haplotype` is verified on a reference basis by the phasing test, which
reads the drawn chromosome off a genotype it constructed itself. That test
also gains a premise guard: without it the claim held in one direction
only, since a run that never left chromosome 0 satisfies every assertion in
it while proving nothing about chromosome 1. `locus` and `subject_id` are
excluded from the denominator and pinned as such -- both are echoed onto
every record rather than sampled per record -- so one field carries the row.

The coverage pin moves deliberately: verified 22 -> 23, no-truth-exported
4 -> 3. It is the second row to leave `no-truth-exported` by the gap
closing rather than by a test being added, and the first where part of the
gap was simply wrong.
The genotype guide gains `to_genotype_tsv` beside `Genotype.to_tsv`: the
long form, one row per carried allele, keyed on `subject_id` so it joins to
the records, with the `(none)` sentinel for a deleted chromosome named where
a reader will meet it.

The benchmarking recipe writes it alongside the AIRR table, since a dataset
handed to someone else should arrive with the truth it is scored against.

There is deliberately no per-read novelty column, so the join is shown
concretely rather than described: read the sidecar's novel pairs into a set,
then test `(subject_id, truth_v_call)` against it. A reader who has to
invent that will add the column themselves, which is exactly what the record
schema is trying to avoid. The same join answers the deletion and zygosity
questions per read. Run end to end before being written down: 27 of 50
records when the planted novel allele is the subject's only V option, 0 when
it is one of 52, which is the right answer in both directions.
…side

A whole-branch review found the sidecar writing a truth file that is a
silent subset of the truth, which is the one failure this module was
written to prevent.

`Genotype.permissive` leaves unspecified genes to be sampled over the whole
reference, unphased, and `to_table` emits a row only per *specified* gene.
So a permissive genotype projected to a sidecar loses every gene nobody
pinned. Measured on HUMAN_IGH_OGRDB: `permissive().with_subject(...)` wrote
a 66-byte header-only file and returned successfully while the records named
five real alleles; with one gene pinned it wrote two rows while the records
drew 119 alleles absent from the file. A benchmark scoring a caller against
that marks correct calls wrong, which is worse than having no file at all.
The writer now refuses, keyed on what is specified rather than on
`is_permissive`, because `permissive().complete_from_reference()` stays
permissive while its table becomes complete. One existing test was built on
a permissive genotype and so was asserting the sentinel inside an already
incomplete table; it is rebuilt on a complete one.

Two neighbouring refusals for the same reason: a cohort where only some
subjects carry a genotype used to drop the Nones and write a table short by
whole subjects, and two subjects sharing one `subject_id` used to merge
under one join key.

The `locus` column's documented property was false. A record carries one
locus for the whole rearrangement, taken from its V call; a row carries the
locus of its own gene. On HUMAN_TCRD_IMGT, whose V pool holds both TRAV and
TRDV genes, 134 of 200 records disagree with the sidecar row for the very
allele they were generated from. The join is on the allele and is unaffected,
so the column is now documented as what it is, with a warning against
filtering by it, and the test class that was named for the records -- and
never looked at one, which is how this survived -- now reads real records on
both a single-locus and a mixed-locus cartridge.

`germline.genotype` keeps a truth_gap, in reduced form. Removing it outright
over-claimed: per-chromosome expression weight is exported nowhere, so a
genotype with weights (1.0, 0.0) projects 85 rows claiming haplotype-1
alleles are carried while no record can draw one. That is the remaining part
of the mechanism's own summary clause. The pin moves to gap-limited, so the
distribution is 22 verified / 4 gap-limited / 3 no-truth / 2 consistent.

Also: the sidecar wrote CRLF beside a records file that pins LF, leaving a
stray carriage return on the last column in any shell pipeline joining the
two; four of the eight columns had no file-level assertion at all, including
the `novel` column whose spelling the documented join compares against the
literal "True"; the deletion question was documented as answerable per read,
which it is not, since a deleted gene is never drawn from; the fully-deleted
premise guard was satisfied by a hemizygous deletion, which would have left
its test asserting nothing; and `Genotype.to_tsv` opened without an explicit
encoding, so a non-ASCII subject_id raised on Windows.
@MuteJester
MuteJester merged commit 23325dc into master Nov 5, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant